{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 与问题\n",
    "\n",
    "考虑如下数据表：\n",
    "\n",
    "|编号| $$x_1$$ | $$x_2$$ | $$d$$ |\n",
    "|:--:| :-----: | :-----: | :---: |\n",
    "|1   |    0    |    0    |   0   |\n",
    "|2   |    1    |    0    |   0   |\n",
    "|3   |    0    |    1    |   0   |\n",
    "|4   |    1    |    1    |   1   |\n",
    "\n",
    "可以看出，这里的运算关系是这样的：\n",
    "$$\n",
    "y = x_1 and x_2\n",
    "$$\n",
    "那么应该如何找到一组参数使得下面的方程满足表中的数据呢？\n",
    "\n",
    "$$\n",
    "y = g(w_1*x_1+w_2*x_2+b)\n",
    "$$\n",
    "\n",
    "----------------------------------------------------------------------------------------------------------------------------------------------\n",
    "解：\n",
    "y=g(x) g为阶跃函数，x=w_1*x_1+w_2*x_2+b，所以有：\n",
    "w_1*0+w_2*0+b <= 0 -》         b <= 0\n",
    "w_1*1+w_2*0+b <= 0 -》 w_1+    b <= 0\n",
    "w_1*0+w_2*1+b <= 0 -》     w_2+b <= 0\n",
    "w_1*1+w_2*1+b >  0 -》 w_1+w_2+b >  0\n",
    "满足条件的w有无数个，如：\n",
    "1) b=-9,w_1=5,w_2=6 -》 y = g(5*x_1+6*x_2-9)\n",
    "2) b=-2,w_1=1,w_2=2 -》 y = g(1*x_1+2*x_2-2)\n",
    "3) b=-5,w_1=4,w_2=2 -》 y = g(4*x_1+2*x_2-5)\n",
    "...\n",
    "\n",
    "对机器来说，一般采取梯度下降法不断更新w和b：\n",
    "1)定义损失函数:L(w,b)=(y-d)(xw+b);\n",
    "2)其对w梯度为:grad_w=(y-d)x;b的梯度:grad_b=(y-d);\n",
    "3)对权重更新时使用：w_new=w_old+a(t-y)x;b_new=b_old+a(d-y);\n",
    "4)不断迭代，至误差小于目标或者w,b不再更新。"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 或问题\n",
    "\n",
    "考虑如下数据表：\n",
    "\n",
    "| 编号 | $$x_1$$ | $$x_2$$ | $$d$$ |\n",
    "| :--: | :-----: | :-----: | :---: |\n",
    "|  1   |    0    |    0    |   0   |\n",
    "|  2   |    1    |    0    |   1   |\n",
    "|  3   |    0    |    1    |   1   |\n",
    "|  4   |    1    |    1    |   1   |\n",
    "\n",
    "可以看出，这里的运算关系是这样的：\n",
    "$$\n",
    "y = x_1 or x_2\n",
    "$$\n",
    "\n",
    "----------------------------------------------------------------------------------------------------------------------------------------------\n",
    "解：\n",
    "y=g(x) g为阶跃函数，x=w_1*x_1+w_2*x_2+b，所以有：\n",
    "w_1*0+w_2*0+b <= 0 -》         b <= 0\n",
    "w_1*1+w_2*0+b >  0 -》 w_1+    b >  0\n",
    "w_1*0+w_2*1+b >  0 -》     w_2+b >  0\n",
    "w_1*1+w_2*1+b >  0 -》 w_1+w_2+b >  0\n",
    "满足条件的w有无数个，如：\n",
    "1) b=-1,w_1=2,w_2=3 -> y = g(2*x_1+3*x_2-1)\n",
    "2) b=-4,w_1=5,w_2=6 -> y = g(5*x_1+6*x_2-4)\n",
    "3) b= 0,w_1=1,w_2=2 -> y = g(8*x_1+9*x_2-0)\n",
    "...\n",
    "\n",
    "对机器来说，一般采取梯度下降法不断更新w和b：\n",
    "1)定义损失函数:L(w,b)=(y-d)(xw+b);\n",
    "2)其对w梯度为:grad_w=(y-d)x;b的梯度:grad_b=(y-d);\n",
    "3)对权重更新时使用：w_new=w_old+a(t-y)x;b_new=b_old+a(d-y);\n",
    "4)不断迭代，至误差小于目标或者w,b不再更新。"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "课程中已经介绍了感知器，本作业要求学员运行下述感知器的代码，感受感知器的运作过程。代码如下:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [],
   "source": [
    "import numpy as np\n",
    "\n",
    "# 逻辑与数据\n",
    "samples_and = [[0, 0, 0],[1, 0, 0],[0, 1, 0],[1, 1, 1]]\n",
    "\n",
    "# 逻辑或数据\n",
    "samples_or = [[0, 0, 0],[1, 0, 1],[0, 1, 1],[1, 1, 1]]\n",
    "\n",
    "#逻辑异或数据\n",
    "samples_xor = [[0, 0, 0],[1, 0, 1],[0, 1, 1],[1, 1, 0]]\n",
    "\n",
    "#感知器的学习，梯度下降法求w，迭代10次\n",
    "def perceptron(samples):\n",
    "    w = np.array([1, 2])\n",
    "    b = 0\n",
    "    a = 1\n",
    "\n",
    "    for i in range(10):\n",
    "        for j in range(4):\n",
    "            x = np.array(samples[j][:2])\n",
    "            y = 1 if np.dot(w, x) + b > 0 else 0\n",
    "            d = np.array(samples[j][2])\n",
    "            \n",
    "            delta_b = a*(d-y)\n",
    "            delta_w = a*(d-y)*x\n",
    "\n",
    "            print('epoch {} sample {}  [{} {} {} {} {} {} {}]'.format(i, j, w[0], w[1], b, y, delta_w[0], delta_w[1], delta_b))\n",
    "            w = w + delta_w\n",
    "            b = b + delta_b"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "代码运行截图中，对训练数据samples_and的训练结果，输出的前半部分如下："
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "logical and\n",
      "epoch 0 sample 0  [1 2 0 0 0 0 0]\n",
      "epoch 0 sample 1  [1 2 0 1 -1 0 -1]\n",
      "epoch 0 sample 2  [0 2 -1 1 0 -1 -1]\n",
      "epoch 0 sample 3  [0 1 -2 0 1 1 1]\n",
      "epoch 1 sample 0  [1 2 -1 0 0 0 0]\n",
      "epoch 1 sample 1  [1 2 -1 0 0 0 0]\n",
      "epoch 1 sample 2  [1 2 -1 1 0 -1 -1]\n",
      "epoch 1 sample 3  [1 1 -2 0 1 1 1]\n",
      "epoch 2 sample 0  [2 2 -1 0 0 0 0]\n",
      "epoch 2 sample 1  [2 2 -1 1 -1 0 -1]\n",
      "epoch 2 sample 2  [1 2 -2 0 0 0 0]\n",
      "epoch 2 sample 3  [1 2 -2 1 0 0 0]\n",
      "epoch 3 sample 0  [1 2 -2 0 0 0 0]\n",
      "epoch 3 sample 1  [1 2 -2 0 0 0 0]\n",
      "epoch 3 sample 2  [1 2 -2 0 0 0 0]\n",
      "epoch 3 sample 3  [1 2 -2 1 0 0 0]\n",
      "epoch 4 sample 0  [1 2 -2 0 0 0 0]\n",
      "epoch 4 sample 1  [1 2 -2 0 0 0 0]\n",
      "epoch 4 sample 2  [1 2 -2 0 0 0 0]\n",
      "epoch 4 sample 3  [1 2 -2 1 0 0 0]\n",
      "epoch 5 sample 0  [1 2 -2 0 0 0 0]\n",
      "epoch 5 sample 1  [1 2 -2 0 0 0 0]\n",
      "epoch 5 sample 2  [1 2 -2 0 0 0 0]\n",
      "epoch 5 sample 3  [1 2 -2 1 0 0 0]\n",
      "epoch 6 sample 0  [1 2 -2 0 0 0 0]\n",
      "epoch 6 sample 1  [1 2 -2 0 0 0 0]\n",
      "epoch 6 sample 2  [1 2 -2 0 0 0 0]\n",
      "epoch 6 sample 3  [1 2 -2 1 0 0 0]\n",
      "epoch 7 sample 0  [1 2 -2 0 0 0 0]\n",
      "epoch 7 sample 1  [1 2 -2 0 0 0 0]\n",
      "epoch 7 sample 2  [1 2 -2 0 0 0 0]\n",
      "epoch 7 sample 3  [1 2 -2 1 0 0 0]\n",
      "epoch 8 sample 0  [1 2 -2 0 0 0 0]\n",
      "epoch 8 sample 1  [1 2 -2 0 0 0 0]\n",
      "epoch 8 sample 2  [1 2 -2 0 0 0 0]\n",
      "epoch 8 sample 3  [1 2 -2 1 0 0 0]\n",
      "epoch 9 sample 0  [1 2 -2 0 0 0 0]\n",
      "epoch 9 sample 1  [1 2 -2 0 0 0 0]\n",
      "epoch 9 sample 2  [1 2 -2 0 0 0 0]\n",
      "epoch 9 sample 3  [1 2 -2 1 0 0 0]\n"
     ]
    }
   ],
   "source": [
    "print('logical and')\n",
    "perceptron(samples_and)\n",
    "\n",
    "#可以看到，最终在epoch2的样本2,3上，参数就已经停止更新了，接下来的epoch3中，参数完全没有更新。\n",
    "#最终得到的参数为$$w_1=1,w_2=2,b=-2$$"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "以与问题同样的参数初始化和训练步骤，输出log前半部分应为：\n",
    "\n",
    "```"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "logical or\n",
      "epoch 0 sample 0  [1 2 0 0 0 0 0]\n",
      "epoch 0 sample 1  [1 2 0 1 0 0 0]\n",
      "epoch 0 sample 2  [1 2 0 1 0 0 0]\n",
      "epoch 0 sample 3  [1 2 0 1 0 0 0]\n",
      "epoch 1 sample 0  [1 2 0 0 0 0 0]\n",
      "epoch 1 sample 1  [1 2 0 1 0 0 0]\n",
      "epoch 1 sample 2  [1 2 0 1 0 0 0]\n",
      "epoch 1 sample 3  [1 2 0 1 0 0 0]\n",
      "epoch 2 sample 0  [1 2 0 0 0 0 0]\n",
      "epoch 2 sample 1  [1 2 0 1 0 0 0]\n",
      "epoch 2 sample 2  [1 2 0 1 0 0 0]\n",
      "epoch 2 sample 3  [1 2 0 1 0 0 0]\n",
      "epoch 3 sample 0  [1 2 0 0 0 0 0]\n",
      "epoch 3 sample 1  [1 2 0 1 0 0 0]\n",
      "epoch 3 sample 2  [1 2 0 1 0 0 0]\n",
      "epoch 3 sample 3  [1 2 0 1 0 0 0]\n",
      "epoch 4 sample 0  [1 2 0 0 0 0 0]\n",
      "epoch 4 sample 1  [1 2 0 1 0 0 0]\n",
      "epoch 4 sample 2  [1 2 0 1 0 0 0]\n",
      "epoch 4 sample 3  [1 2 0 1 0 0 0]\n",
      "epoch 5 sample 0  [1 2 0 0 0 0 0]\n",
      "epoch 5 sample 1  [1 2 0 1 0 0 0]\n",
      "epoch 5 sample 2  [1 2 0 1 0 0 0]\n",
      "epoch 5 sample 3  [1 2 0 1 0 0 0]\n",
      "epoch 6 sample 0  [1 2 0 0 0 0 0]\n",
      "epoch 6 sample 1  [1 2 0 1 0 0 0]\n",
      "epoch 6 sample 2  [1 2 0 1 0 0 0]\n",
      "epoch 6 sample 3  [1 2 0 1 0 0 0]\n",
      "epoch 7 sample 0  [1 2 0 0 0 0 0]\n",
      "epoch 7 sample 1  [1 2 0 1 0 0 0]\n",
      "epoch 7 sample 2  [1 2 0 1 0 0 0]\n",
      "epoch 7 sample 3  [1 2 0 1 0 0 0]\n",
      "epoch 8 sample 0  [1 2 0 0 0 0 0]\n",
      "epoch 8 sample 1  [1 2 0 1 0 0 0]\n",
      "epoch 8 sample 2  [1 2 0 1 0 0 0]\n",
      "epoch 8 sample 3  [1 2 0 1 0 0 0]\n",
      "epoch 9 sample 0  [1 2 0 0 0 0 0]\n",
      "epoch 9 sample 1  [1 2 0 1 0 0 0]\n",
      "epoch 9 sample 2  [1 2 0 1 0 0 0]\n",
      "epoch 9 sample 3  [1 2 0 1 0 0 0]\n"
     ]
    }
   ],
   "source": [
    "print('logical or')\n",
    "perceptron(samples_or)\n",
    "\n",
    "#可以得到\n",
    "#$$w_1=1,w_2=2,b=0$$\n",
    "#事实上，这里比较巧合，我们初始化使用的参数，正好满足这里的数据。"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 异或问题"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "logical xor\n",
      "epoch 0 sample 0  [1 2 0 0 0 0 0]\n",
      "epoch 0 sample 1  [1 2 0 1 0 0 0]\n",
      "epoch 0 sample 2  [1 2 0 1 0 0 0]\n",
      "epoch 0 sample 3  [1 2 0 1 -1 -1 -1]\n",
      "epoch 1 sample 0  [0 1 -1 0 0 0 0]\n",
      "epoch 1 sample 1  [0 1 -1 0 1 0 1]\n",
      "epoch 1 sample 2  [1 1 0 1 0 0 0]\n",
      "epoch 1 sample 3  [1 1 0 1 -1 -1 -1]\n",
      "epoch 2 sample 0  [0 0 -1 0 0 0 0]\n",
      "epoch 2 sample 1  [0 0 -1 0 1 0 1]\n",
      "epoch 2 sample 2  [1 0 0 0 0 1 1]\n",
      "epoch 2 sample 3  [1 1 1 1 -1 -1 -1]\n",
      "epoch 3 sample 0  [0 0 0 0 0 0 0]\n",
      "epoch 3 sample 1  [0 0 0 0 1 0 1]\n",
      "epoch 3 sample 2  [1 0 1 1 0 0 0]\n",
      "epoch 3 sample 3  [1 0 1 1 -1 -1 -1]\n",
      "epoch 4 sample 0  [0 -1 0 0 0 0 0]\n",
      "epoch 4 sample 1  [0 -1 0 0 1 0 1]\n",
      "epoch 4 sample 2  [1 -1 1 0 0 1 1]\n",
      "epoch 4 sample 3  [1 0 2 1 -1 -1 -1]\n",
      "epoch 5 sample 0  [0 -1 1 1 0 0 -1]\n",
      "epoch 5 sample 1  [0 -1 0 0 1 0 1]\n",
      "epoch 5 sample 2  [1 -1 1 0 0 1 1]\n",
      "epoch 5 sample 3  [1 0 2 1 -1 -1 -1]\n",
      "epoch 6 sample 0  [0 -1 1 1 0 0 -1]\n",
      "epoch 6 sample 1  [0 -1 0 0 1 0 1]\n",
      "epoch 6 sample 2  [1 -1 1 0 0 1 1]\n",
      "epoch 6 sample 3  [1 0 2 1 -1 -1 -1]\n",
      "epoch 7 sample 0  [0 -1 1 1 0 0 -1]\n",
      "epoch 7 sample 1  [0 -1 0 0 1 0 1]\n",
      "epoch 7 sample 2  [1 -1 1 0 0 1 1]\n",
      "epoch 7 sample 3  [1 0 2 1 -1 -1 -1]\n",
      "epoch 8 sample 0  [0 -1 1 1 0 0 -1]\n",
      "epoch 8 sample 1  [0 -1 0 0 1 0 1]\n",
      "epoch 8 sample 2  [1 -1 1 0 0 1 1]\n",
      "epoch 8 sample 3  [1 0 2 1 -1 -1 -1]\n",
      "epoch 9 sample 0  [0 -1 1 1 0 0 -1]\n",
      "epoch 9 sample 1  [0 -1 0 0 1 0 1]\n",
      "epoch 9 sample 2  [1 -1 1 0 0 1 1]\n",
      "epoch 9 sample 3  [1 0 2 1 -1 -1 -1]\n"
     ]
    }
   ],
   "source": [
    "print('logical xor')\n",
    "perceptron(samples_xor)\n",
    "\n",
    "#另外，在代码中，其实还提供了一组异或数据。但是，在这里的代码无论怎么写都无法得到满足这组数据的参数。请学员尝试一下，并解释为什么这里的代码无法完成带异或功能的感知器。"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "感知器的局限:\n",
    "1)仅能做0-1输出;\n",
    "2)仅能处理线性分类问题，也即：无法处理异或(XOR)问题。\n",
    "解释为什么这里的感知器代码无法完成异或功能？\n",
    "\n",
    "----------------------------------------------------------------------------------------------------------------------------------------------\n",
    "解：\n",
    "异或是线性不可分的，他不像“与”“或”逻辑运算那样，可以在x1、x2轴直角坐标系上，找到一条直线将结果的“真”和“假”这两类分开，即感知机无法找到一个线性模型对异或问题进行划分。（不光感知机无法处理异或问题，所有的线性分类模型都无法处理异或分类问题）\n",
    "\n",
    "\n",
    "*可以通过添加隐层，组成多层感知机网络，拟合“异或”功能，因为多层感知机能拟合任何多边形超平面切割分类。"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.7.0"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}
